Papers by Thanh Hong Nguyen
MTA: Multi-Granular Trajectory Alignment for Large Language Model Distillation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for reducing the computational cost of large language models (LLMs) focus on minimizing the divergence between the output probability distributions of the teacher and the student, which limits knowledge transfer. |
| Approach: | They propose a framework that aligns teacher and student representations along their layer-wise transformation trajectory. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on teacher–student layers. |
MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining (2026.findings-acl)
Copied to clipboard
Phung Gia Huy, Hai An Vu, Minh-Phuc Truong, Thang Duc Tran, Linh Ngo Van, Thanh Hong Nguyen, Trung Le
| Challenge: | Existing approaches to train dense representations require explicit coordination of how information is arranged across embedding dimensionality and model depth. |
| Approach: | They propose a framework that trains Matryoshka representations using self-distilled intra-relational alignment and Progressive information chaining. |
| Outcome: | The proposed framework produces coherent and compact Matryoshka representations with significant performance advantages under low-dimensional models. |